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ABSTRACT 

In this study, the effect of web-based homework on university students’ physics achievement was compared. 
One of the two identical sections of introductory physics course students received pen-and-paper homework 
done in groups while the other received web-based online homework performed individually. And then both 
groups’ homework performance and achievements were compared by homework assignments and standardized 
test scores (like Force Concept Inventory and Conceptual Survey of Electricity and Magnetism). Although there 
was not any significant difference in standardized test score results, in the first (fall) semester, pen-and-paper 
homework group performed better than the web-based group; yet, the web-based homework group did better in 
the second (spring) semester. 

Keywords: Web-based homework, pen-and-paper homework, physics achievement, first year university 
students. 

INTRODUCTION 

The rise of the Internet in the last decade has led to become an important means for disseminating various 
educational materials to students. As part of this trend, a number of software packages have been developed, 
which allow students to complete homework assignments online. Student's assessment is one of the most 
important elements in an education system. The purpose of assessment is to provide a measure of student 
performance and a context for improving a course or an academic program. The two assessment (self and 
peer/groups) methods are becoming more pervasive and dominant (Paris & Paris, 2001; Shepard, 2000). 

The importance of homework and the relationship between homework and academic performance has already 
been recognized by many individual and meta-analytic studies (Cooper, 1989; Cooper, Lindsay, Nye, & 
Greathouse, 1998; Keith & Cool, 1992; Warton, 2001). Homework is an activity related to motivation, mastery 
of material, and to achievement (Keith & Benson, 1992; Keith, 1982; Paschal, Weinstein, & Walberg, 1984). It 
is also obvious from some studies that homework may be necessary but not satisfactory for achievement on 
exams (Peters, Kethley & Bullington, 2002; Porter & Riley, 1996). 

Some researchers stated that electronic homework as a course element has more positive effects than written 
homework (Dufresne, Mestre, Hart, & Rath 2002; Ogilve, 2000; Thoennessen and Harrison, 1996). For example, 
Dufresne et al. (2002) compared the effect of electronic homework and written homework on student 
achievement found that electronic homework led to higher overall exam performance. However, the study 
conducted by Bonham et al. (2001, 2003) showed that no significant differences in student performance that 
could be attributed to the homework method used. 

Web-based homework has some benefits. These benefits include obtaining students' results faster, having the 
ability to place grades into an electronic format, measuring learning accurately, focusing on a student-centered 
environment, and costing less in comparison to pen-and-paper homework (Bartlett, Reynolds, & Alexander, 
2000; Dash, 2000; Oregon to Administer, 2001). On the other hand, using a pen-and-paper homework has some 
limitations (for example: recording, scoring, getting immediate feedback, etc.). The possible constraints may be 
compensated by technology. Several studies (for example: Bonham, Beichner, & Deardorff, 2003; Dufresne, 
Mestre, Hart, & Rath, 2002; Toback, Mershin, & Novikova, 2005) have reported mixed findings on student 
homework performance, but little on student homework preferences. Therefore, the present study attempts to 
determine the effect of web-based homework on university students’ physics achievements and to give new 
insight into the physics educator to use homework in their lectures. 

Testing and Online Homework 

Testing in general is used for providing feedback and evaluation. Feedback refers to the response regarding a 
critical analysis of students' work. Evaluation refers to the grading and recording of students' work for assessing 
their understanding of the material. A testing instrument, whether it is a homework assignment, quiz, exam, or 
practice test, can satisfy both purposes to a varying degree. Evaluation and feedback have different goals and 
thus have different implementation requirements. Evaluation is primarily used to record student responses and 
assign grades, security concerns such as verifying a student's identity, protecting answer keys, limiting access 
according to a specific time or location, and preventing unauthorized sharing of information need to be 
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considered. Feedbaek is to be able to respond to students' input by providing "correct/incorreet" answers, hints, 
and solutions or by engaging the student in additional learning aetivities much like present study. 

The online homework method has some advantages for the instructor as well (Johnston, 2002). The primary 
benefit is savings in the effort spent running the coordination of the pen-and-paper homework process. Copying, 
distributing, collecting, sorting, and filing homework documents are freed by the instructor. Educators should 
consider and evaluate the effeetiveness of this teehnology for inereasing student learning. Homework 
assignments must exist in education because it might increase students’ interest in school and/or course topics 
and improve their academic development (Altun, 2008). The online method also eliminates the need to seore 
homework and record the scores in a grade book. The problems of missed, lost, and submitted-late assignments 
are largely eliminated. The online approaeh automates the tasks. Moving these tedious tasks outside of the class 
time period enables an instructor to cover more things in classrooms. 

Purpose 

The main aim of this study was to eompare the effect of web-based homework and pen-and-paper homework on 
university students’ physics achievement as measured by exam and homework performance. 

METHODOLOGY 

Participants 

The participants of this study were ehosen from a sample of convenience of Computer Edueation Department. 
They have many experiences online and computer-based assessment methods. In order to eliminate novelty 
effeet, any kind of aetivities in the study has to be not new to students. In an introduetory physics course, all 
students were taught partially “peer instruction” method (adopted from Mazur, 1997; Fagen, Crouch, & Mazur, 
2002). In the study, there were two identieal elasses (according to their pretest seores). One class used online 
homework system and the other class used pen-and-paper homework. Specifically, 41 students in the fall 
semester of 2005, and 48 students in the spring semester of 2006 used the Web-based homework system; 37 
students in the fall semester of 2005, and 42 students in the spring semester of 2006 used the pen-and-paper 
based homework. 

Design and Procedures 

A two-group pretest-posttest quasi-experimental design was used in this study. One group is subjected to a 
treatment, and the other is subjected to a control group (Fraenkel & Wallen, 1996). Students were registered for 
the two different seetions through a standard eourse registration system and were unaware of the homework 
method until they were announced that they were assigned the indicated homework group (the first week of 
physies-1 [in the fall semester) and physics-2 ( in the spring) elasses, respeetively]. The physies-1 and -2 eourses 
have two main exams, one of which is mid-term and the other is the final exam. The homework performance 
seores in both groups were added to include the 20 % of the final grade of the eourse for eaeh semester. The 
treatment group received their homework via an online quiz system where it was automatically graded by the 
software. Control group wrote out solutions to homework exereises on paper with working as groups consisted 
of four to five students. These exercises were turned in and graded by the instructor. Through semesters 
completing the each unit; homework assignment was administrated to students aceording to their assigned 
method (pen-and-paper or online) (mainly these assignment questions derived from Turkish translation of 
Principles of Physics by Bueche and Jerde, sixth edition, 1995). There were eight-homework exercises in the fall 
semester (for physics- 1) and six exercises in the spring semester (for physics-2). All results were graded in 
percent scores, and then average scores were calculated to be used as the homework performance score. In the 
fall semester. The Force Concept Inventory (FCI) test (Hestenes, Wells, & Swaekhamer, 1992) administered 
before (in the first week of the semester) and after the instmction (the last second week semester). In the spring 
semester, also The Concept of Survey on the Electricity and Magnetism (CSEM) test (Maloney, O’Kuma, 
Hieggelke, & Heuvelen, 2001) was administered before (in the first week of the semester) and after (the last 
seeond week of the semester) the instruction to both groups. FCI and CSEM tests scores (in percent average 
scores) and homework performance scores were entered into the SPSS package programme for statistical 
analyses. 

Structure of groups 

Pen-and- paper homework group consisted of four or five students per group. There were 37 students in the fall 
semester and 42 students in the spring semester. Heterogeneous groups (aecording to prior knowledge) were 
formed to provide students with opportunities to interact with the content through the varying perspectives of 
their peers. According to Mahendra, Bayles, Tomoeda, and Kim (2005), promoting a collaborative, supportive 
classroom culture increases the opportunities for learners to be exposed to diverse viewpoints and values. 
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Students in this group worked with their group memhers and turned in their homework at the end of each unit hy 
the end of each semester. 

The types of problems used in this group homework assignment were identical to those used in weh-hased 
homework assignments; in fact, the vast majority of problems in the web based homework library came from the 
end-of-the chapter problems of the standard first year university algebra-based physics textbooks with the 
addition to some conceptual questions. After finishing every unit —there were eight units in the fall semester and 
six units in the spring semester—, students were given one assignment of five to nine problems. Gibson, Tesone, 
and Blachwell, (2001) suggested that the online testing should be one component of the evaluation of the 
student; therefore, in both groups, 20 % of the course final grade comes from the average homework score. In 
the web-based homework group, each student first registered the system and did their homework via online 
individually. The web-based homework system is called “online testing” and detailed description of the program 
is given below. 

The Web-based homework system 

The goal of the web-based homework system called “online testing” is to get immediate “correct” or “incorrect” 
feedback. The “online testing” system did not correct the student’s errors or give them hints. The Web-based 
homework system was developed by Linux based .php extension html environment with using the MySQL 
database system and has two main modules. The first module is for students, and the second is for teachers. 

In students’ module, students have to register into a password protected web-based homework system at the 
beginning of the course. Until the end of the semester, they only need their ID number and password to log into 
the system. Once they enter the system whenever homework is activated by the instructor after each unit, one 
would be able to take that test. After finishing that test, students could see their results immediately and get the 
feedback. Student may ask any question related to testing or any problem encountered any time just clicking the 
provided link to communicate with the instructor via e-mail. 

The teacher module provides teachers with a convenient user interface that allows them to execute various setup 
and management functions online, such as setting up accounts, setting up test parameters, queries as to students’ 
scoring process and observing various assessment results at any time. Web-based homework system offers 
parameters to configure the options of various types of activities. By the teacher module, the instmctor might do 
the following main tasks; 

• Create or delete homework assignments, quizzes and define the number of questions to be asked for 
each assignment. 

• To see students’ homework results and progress, and see their detailed assignment results such as 
starting and finishing time. 

• Answer their e-mail messages to communicate with them to solve their problems they might have 
encountered during the process. 

• Able to activate or deactivate any particular assignment. 

• Define the level of difficulties of questions uploaded to a server (specifically based on Bloom’s 
taxonomy, questions in cognitive, knowledge and conceptual level labeled as “easy”; questions in 
application level labeled as “normal”; questions in complex application and analysis level labeled as 
“difficuh”). 

RESULTS 

This part consists of two sections, descriptive statistics, and inferential statistics. 

Descriptive Statistics 

Results devoted to FCI and CSEM tests 

The results of FCI pre-, posttest scores, normalized-gain scores also known as Hake factor (Hake, 1998), and 
homework performance scores (in percent) for both groups are given in Table 1 . 


Table 1 The results of FCI Tests and Homework Performance Scores in Percentage 

Web-based homework-group Pen-and-paper homework group 



n 

Average 

S. Deviation 

n 

Average 

S. Deviation 

Homework performance 

41 

71.15 

15.428 

37 

80.30 

7.237 

FCI pre-test 

40 

41.05 

11.89 

33 

42.73 

9.69 

FCI post-test 

39 

62.87 

9.96 

36 

61.44 

9.97 


FCI normalized gain %37.01 %32.66 
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It can be seen in Table 1 that average FCI pretest score calculated (in percent) in both groups are about in the 
forties and posttest score are about in the sixties. However, homework performance (percent) score calculated for 
web-based group is about 71%, and for pen-and-paper group is about 80 %. While web-based homework group’s 
FCI normalized gain score is about 37 %, the pen-and-paper-based homework group’s gain scores are found 
about 32.6 %. Pen-and paper group got higher scores on homework performance, but lower on the normalized 
gain score than the web-based homework group in the fall semester. 

The results of CSEM pre-, posttest scores, normalized-gain scores, and homework performance scores (in 
percent) for both groups are given in Table 2. 


Table 2 Results of he CSEM Tests and Homework Performance Scores in Percentage 



Web-based homework- 

•group 

Pen-and- 

-paper homework group 


N 

Average 

S. Deviation 

N 

Average 

S. Deviation 

Homework performance 

48 

78.13 

8.41 

42 

71.66 

10.60 

CSEM pre-test 

42 

31.40 

11.15 

31 

31.25 

9.24 

CSEM post-test 

42 

53.79 

17.19 

31 

53.43 

8.12 

CSEM normalized gain 


%32.63 



%31.36 



Inferential Statistics 

Summary of the t-test results related to FCI and CSEM pre- and post-test scores and homework performance 
scores for both groups are given in Table 3. 


Table 3 t-test Summary Results 


Test differences between groups 

df 

t-test 

P 

Homework performance differences for physics- 1 

76 

-3.29 

0.002 

FCI pre-test differences 

71 

-0.65 

0.517 

FCI post-test differences 

73 

0.61 

0.539 

Homework performance differences for physics-2 

88 

3.08 

0.003 

CSEM pre-test differences 

71 

-0.264 

0.793 

CSEM post-test differences 

71 

0.153 

0.879 


CONCLUSION AND DISCUSSIONS 

There was not any statistically significant difference were found in the means of the web-based and grouped pen- 
and-paper homework with respect to FCI and CSEM pre- and posttest scores. Although there was a significant 
difference in students’ homework performance scores in favor of the pen-and-paper group in the fall semester, 
this was changed in the spring semester to the web-based group. Bonham, Beichner, and Deardorff (2001, 2003) 
reported that students who used traditional hand-written homework experienced no significant differences in 
learning gains comparing to students using the WebAssign system. Dufresne, Mestre, Hart, and Rath, (2002) 
compared student performance over several years in large introductory physics courses with both pen-and-paper- 
based and web-based homework system, and they found similar results from the study done by Ployhart et al. 
(2003) suggesting that automated homework led to higher performance scores. 

Studies carried out by the CAPA development team concluded that online homework had a significant, positive 
effect on student learning (Demirci, 2007; Kashy, Sherrill, Tsai, Weinshank, Englemann, & Morrissey, 1993; 
Kashy, Morrissey, Tsai, & Wolfe, 1995; Morrisey, Kashy, & Tsai, 1995; Kashy, Thoennessen, Tsai, Davis, & 
Wolfe, 1998). 

Web-based homework is a possible alternative to the traditional pen-and-paper -based approach. It does not 
bring significantly greater advantage to the students, but neither does it work much worse than standard methods 
of collecting and grading homework. This supports the opinion that technology itself does not improve or 
decrease student learning. Automated homework system most likely will help students in courses where 
homework could not otherwise be assigned. 

The current literature does not really answer questions being raised about pen-and-paper-based web-based or 
otherwise. Homework is important in technical courses such as introductory physics, where problem solving is a 
major focus, and homework is the main place for practicing. Many student struggles to develop problem-solving 
skills in physics (Maloney, 1994), although directed instruction and feedback has been shown to be effective 
(Heller & Reif, 1984; Heller & Hollabaugh, 1992). 
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A limitation of the study was that online homework was done individually while the pen-paper based homework 
was done eollaboratively with groups of 4/5 students. Individual versus eollaborative work itself can be the 
conditions for other experimental studies. Furthermore, FCI and CSEM test scores thought to be fits and an 
adequate for measuring students’ conceptual understanding and physics achievement. The online course cannot 
be an example of good pedagogy if technology is used without a student-centered approach to learning (Hiltz, 
1990). When online material is submitted, and when it becomes an important student-centered activity and 
requirement of the class, Bauer and Anderson (2000) recommend three criteria to judge the students’ content, 
expression, and participation. “These can provide a unique perspective from which to view students’ formal 
writings and informal discussions”. Besides, “an e-folio can offer students a chance to reflect on their own work 
and thus become more involved in the assessment process” (Bauer and Anderson, 2000, p.70). 

For future study, it would be fruitful to examine such behaviors in relation to other factors associated with 
learning, such as students’ attitudes, possible environmental variables, and different learning strategies and 
methods. Further work could focus on how online-mediated evaluation affects distance and open learning, how 
the development of content delivery and assessment tools is directly related to learning styles, how feedback in 
e-leaming environments can be enriched and finally how the evaluation of e-leaming methods differs from more 
traditional ones in great details. Additional contextual factors could reflect on students’ some extra-curricular 
activities. Such inquiries could prove to yield some beneficial results (Kotas & Finck, 2002). 
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